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ABSTRACT 



This paper discusses the results of a study that 
investigated the reliability and validity of the Youth Client Satisfaction 
Questionnaire (YCSQ) . The YCSQ is administered as a telephone interview, 
which provides a higher response rate than mailed surveys. Participants in 
the study were 87 clients (ages 11-17) and their parents who had completed 
therapy at a community mental health center for children in northeastern 
Ohio. The sample included only subjects who had participated in at least 
three therapy sessions. Telephone administration of the YCSQ usually took 
about 10 minutes unless there was spontaneous elaboration. One -half of the 
subjects responded to the questionnaire again in three weeks to assess 
reliability. Results of the survey indicate that the YCSQ is a reliable, 
internally consistent, and valid measure of satisfaction of mental health 
services for young clients, however, more research is urged. The 14 rated 
items on the survey demonstrated satisfactory psychometric properties and an 
interpretable factor structure, suggesting that the relationship with the 
therapists and perceived benefits of counseling are two main components of 
overall youth satisfaction. (CR) 
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Introduction 

Current thinking about the child mental health system 
emphasizes the importance of consumer satisfaction and a 
child-centered approach (Knitzer, 1982; Stroul & Friedman, 
1986). Successful implementation of these principles 
requires a reliable and valid method of producing 
differentiated information about young people's satisfaction 
with their services. Efforts to develop such methods have 
only recently begun. Existing measures have been 
characterized as being limited by a lack of psychometric 
information, sampling problems, vulnerability to social 
desirability response bias, and a lack of differentiation 
among dimensions of satisfaction (Young, Nicholson, & 
Davis, 1995). 

The Youth Client Satisfaction Questionnaire (YCSQ) was 
designed to differ from existing measures of this construct 
by combining several characteristics. First, this instrument 
is administered as a telephone interview, which provides a 
higher response rate than mailed surveys (Young et al., 
1995). The YCSQ also provides differentiated information 
about dimensions of client satisfaction, and its directions 
and item wording are designed to minimize social 
desirability response bias. 
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Participants were 150 clients and their parents who had 
completed therapy at a community mental health center for 
children in northeastern Ohio. The age range was 11-17 and 
the mean age was 13.23 (SD = 1.72). The sample was 51% 
female. Ethnic composition was 43% African American, 
50% White, 5% Hispanic, and 2% other ethnic group. The 
sample included only subjects who had participated in at 
least three therapy sessions (not counting the initial 
diagnostic assessment). 

The primary income source was employment for 37% of the 
families, with 63% receiving AFDC or another form of 
income assistance. A total of 62% of the families were 
headed by a single parent, and 38% of the families provided 
two-parent homes. Eighteen percent of the youth had a 
history of juvenile court involvement. At intake, 28% of the 
sample were diagnosed with Oppositional-Defiant Disorder, 
8% with Adjustment Disorder with Mixed Emotional 
Features, 8% with Sexual Disorder Not Otherwise Specified 
(juvenile sex offenders), 6% with Dysthymic Disorder, and 
the remainder widely dispersed among a number of 
diagnoses. 

Most subjects participated in a combination of treatment 
modalities, so that the following proportions add up to more 
than 100%. Seventy-seven percent of the subjects received 
individual therapy, 91% family therapy, and 22% group 
therapy. 

Fifty-eight percent of the consumers eligible for the study 
became participants (N = 87). Only 34% of the attrition was 
due to client unwillingness to participate. The remainder of 
non-participants resulted from the family not having a 
phone, moves without forwarding information, the family's 
inability to complete the interviews by eight weeks from 
their last appointment, language barriers, and the child no 
longer being in the home. 

Candidate items for the YCSQ were generated on the basis 
of a literature review, focus groups with clinicians, and 
individual exploratory interviews with 1 0 young clients and 
their parents. This resulted in 1 7 candidate items. The 
design of the YSCQ involved combining psychometric 
considerations with the need to maximize ease and comfort 
of responding. Responses were coded in the form of a 
four-point Likert-type scale. However, rather than 
presenting the scale directly to subjects, responses were 
obtained through very brief, scripted exchanges between 
interviewer and youth, with follow-up questions when 
necessary. The items were given different scale anchors, 
depending on the item content. 

Telephone administration of the YCSQ usually took about 
10 minutes unless there was spontaneous elaboration. 
One-half of the subjects responded to the questionnaire 
again in three weeks to assess reliability. One item was 
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again in three weeks to assess reliability. One item was i 
repeated as a reliability check. Five protocols were excluded | 
because of evidence of random responding. 

The procedure also included the Child Behavior Checklist 
(CBCL; Achenbach, 1991a) and the Youth Self-report 
(YSR; Achenbach, 1991b), which were administered prior 
to and after therapy as an assessment of outcome. Parents 
also completed a measure of their satisfaction with services 
(Kotsopoulos, Elwood, & Oke, 1989) during a telephone 
interview. 

Results 

Because YCSQ items have different anchors, there were 
questions about the parametric versus nonparametric nature 
of the data. In order to address this issue, we computed 
Spearman correlations for the psychometric analyses 
addressed below and compared them to the corresponding 
Pearson correlations. The 5 1 pairs of correlations were 
extremely similar, with no significant differences, 
suggesting that YCSQ scaling design does not result in 
marked departure from interval equality and item 
comparability, so that parametric procedures are appropriate 
for the data. 

There were three criteria for retaining items in the measure: 

(a) test-retest reliability correlation ^ .50; (b) part-whole 
correlation ^ .40; and (c) correlation with a validation item 
about overall satisfaction ^ .30 (see Table IV Fourteen items 
met these criteria. All correlations reported below are 
significant with p < .0001 unless otherwise indicated. The 
average reliability correlation was .67, with an average 
part-whole correlation of .65, and an average validity 
correlation of .48. The item with the highest part-whole 
correlation (.78) was the one about overall satisfaction. 

Total scores had a reliability correlation of .92. Cronbach's 
alpha was .90. 

Total scores on the instrument ranged from 8 to 42, with a 
mean of 30.64 and a standard deviation of 8. 1 1 . The average 
item score was 2.19. Given an item score range of 0 to 3, 
these results indicate that the sample as a whole expressed a 
fairly high degree of satisfaction with their 
services&endash;a finding consistent with the general client 
satisfaction literature (Young et al., 1995). 

We performed a principal components analysis with ! 

varimax rotation in order to identify the major factors of 
youth consumer satisfaction. Two factors emerged. The 
Relationship with Therapist factor included six items asking 
whether the youth liked the counselor, had a good time in 
therapy, perceived the therapist as understanding, caring, 
having helpful ideas, and as understanding his or her 
cultural group. The Benefits of Therapy factor included 
eight items asking about therapy-caused change in the areas 
of feelings, behavior, self-concept, personal problems, and 
family relationships, and asking about beneficial learning. 
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family relationships, and asking about beneficial learning, 
understanding therapeutic goals, and overall satisfaction. 
Cronbach's alpha for the individual factors was .85 in both 
cases. The inter-factor correlation was .66. 

In order to assess convergent validity, we computed 
correlations between the YCSQ and several other measures 
of treatment quality (see Table 2) . YCSQ scores were 
somewhat positively related to improvement from pretest to 
posttest on the CBCL (r = .18, p < .05), but not the YSR (r = 
-.04, ns). YCSQ scores had correlations of .37 with 
improvement in DSM III-R Global Assessment of 
Functioning scores, .41 with a therapist rating of treatment 
progress, .53 with parent satisfaction scores, and .47 with a 
parent rating of treatment progress. 

Boys produced higher YCSQ scores than girls (t = 2.00, p < 
.05). There was a marginally significant correlation with age 
(r = -.15, p < .07). African American and White clients 
produced similar scores (t = .29). Client satisfaction did not 
differ depending on parental employment status (t = 1.67), 
single versus two-parent families (t = .56), past treatment 
history at our agency (t = .58) or another setting (t = .59), or 
youth involvement in the juvenile justice system (t = .52). 
YCSQ scores showed a marginally nonsignificant 
correlation of .16 (p < .06) with number of sessions. 

The interview included two questions asking whether the 
youth wanted to receive counseling or if they were required 
to come. There was no relation between client satisfaction 
and whether the youth chose or were required to begin 
counseling (r = .12). Youth who reported voluntary 
participation in therapy by the time of termination were 
more satisfied with their services (r = .50). 

Conclusions 

The results suggest that the YCSQ is a reliable, internally 
consistent, and valid measure of satisfaction of mental 
health services for young clients, however, more research is 
needed. The 14 retained items demonstrated satisfactory 
psychometric properties and an interpretable factor structure 
suggesting that the relationship with the therapist and the 
perceived benefits of counseling are two main components 
of overall youth satisfaction. Youth client satisfaction 
showed significant relations with 5 of 6 other measures of 
treatment effectiveness (with the exception of the YSR), 
providing evidence of convergent validity for the YCSQ. 

Telephone interviews proved to be a feasible method of 
administration. Given that mailed questionnaires often have 
low return rates and that terminations are often unplanned, 
the telephone method seems to have practical advantages. 
However, youth may not have felt the survey was 
anonymous when conducted by phone by staff from the 
center, leading to bias towards positive responses 
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The descriptive results suggest that most young counseling 
clients express a fairly high level of satisfaction with their 
services even when directions and items wording are 
designed to give permission for dissatisfied as well as 
satisfied responses. Clients who began treatment 
involuntarily ended with the same level of satisfaction as 
youth who initially wanted to receive therapy. Inteipretation 
of these results should be cognizant of the possibility that 
clients who had dropped out prior to their third therapy 
session and those who were not recruited for the study may 
have had a lower level of satisfaction than our study 
participants. 
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